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os GES DISC began as the Goddard 

Distributed Active Archive Center (DAAC) 

08 Ingest, process, store and distribute Earth 
science data (mostly remote sensing) 

oa In the last decade, services have been 
added 

os Discovery 
03 Access-related 




The Data Usage Cycle 
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PREPARE 



Preparation Steps 


oa Subsetting 
os Variable 
os Space 
os Time 

oa Gridding / (re)projection 
oa Reformatting to work in the analysis tools 
oa Quality Filtering 


How much of the Preparation process 
can we build into the Access step? 





On-the-Fly Web Services 
executed on acquisition 


On-the-Fly Web Services 


os REST-like: acquire as URLs 

ca Limits error return possibilities 

oa Requires an HTTP trick (shhh...) for long-running 
processes 

oa Accommodates any executable that... 

oa ...Takes one file as input 
oa ...Produces one file as output 

oa On-the-fly execution means minimal disk 
buffer requirements 

oa No need to stage the whole request for pickup 




On the Fly Subsetting 


Subsetted Data Download Instructions: 

- Multiple file download: 

■ To use wget, you will need version 1.11.1 or later. To find the version, run wget 
V'. 

■ If necessary, obtain the latest version of wget & 

■ Download the list of URLs 

■ Run: wget -content-disposition -i wget khEilxSN 

■ Single file download: click on the link for each file to be downloaded: 

MERRA100.prod.assim.inst3_3d_asm_Cp,1 97901 31 .SUB.nc 
MERRA100.prod.assim.inst3_3d_asm_Cp.1 97901 30. SUB. nc 
MERRA100.prod.assim.inst3_3d_asm_Cp.1 97901 29. SUB.nc 
MERRA100.prod.assim.inst3_3d_asm_Cp.1 97901 28. SUB.nc 
MERRA100.prod.assim.inst3_3d_asm_Cp.1 97901 27. SUB.nc 



NOTE: Default Selection Is All 

rjSLP = Sea-level pressure 

nPS = Surface pressure 

□PHIS = Surface Geopotential 

QH = Geopotentlal Height 

□03 « Ozone mixing ratio 

QiQV = Specific humidity 

QQL = Cloud liquid water mixing ratio 

QQI = Cloud Ice mixing ratio 

rtRH = Relative Humidity 

QT ■ Air Temperature 

= Eastward wind component 
iv*V = Northward wind component 
□EPV = Ertel potential vorticity 
QOMEGA = Vertical pressure velocity 


NOTE: Default Selection is All 


Additional Options 


'Modern Era Retrospective-Analysis for Research and Applications 



On-the-fly Conversion to netCDF 

(network Common Data Form) 

os Most Earth Observing System datasets are in Hierarchical 
Data Format (HDF) 

08 BUT, many visualization tools understand netCDF “better” 



TRMM Monthly Rainfall rate for Oct 2011 in Panoply 
http://www.giss.nasa.gov/tools/panoply/ 






Data Quality Screening Service 


ca Level 2 Satellite data often comes with quality control 
flags 


08 Until now, each user typically had to write his/her own 
SQifh^f>©rti0s|iH^clftadr6|lJality data— or ignore them 
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Hurricane Ike , 9/10/2008 





The Data Quality Screening Service for 
AIRS Level 2 swath data 


Original data 
array 



Total column 
precipitable water 



Mask based on 
user criteria 
Quality flag<2 



Good quality 
data pixels 
retained 



Output file has the same format and structure as the 
input file, with fill values replacing the low-quality data 






OPeNDAP*: 

a protocol standard for remote 

access 


Open-source Project for a Network Data Access Protocol 


OPeNDAP: Subsetting and more 
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View Projections 


Legend 


4MI ► ll> 



Data Attributes 

* 

Data Objects 
< — 


Serve 

r 



os Subsetting 

os individual 
variables 

os slices of variables 


os Reformatting: download 
as... 

03 ASCII 
03 netCDF 




Varieties of OPeNDAP 


oa Hyrax 

os High performance 
oa Reformat to netCDF 

oa GrADS Data Server 
oa Multiple input formats 
oa Server-side processing 

oa THREDDS Data Server 
oa Aggregation 

oa Web Coverage Service, netCDF Subsetter 
oa Others: ERDDAP, PyDAP, Dapper... 



Giovanni: 

online analysis and visualization 


Giovanni 


oa Analysis and visualization server 
oa Workflow paradigm 
oa Steps for: 

oa Fetching 
oa Subsetting 
oa Quality filtering 
oa Regridding 
oa Averaging 
oa Visualization 


oa Output can be downloaded 



Example: Carbon Monoxide from 2010 

Russian wildfires 



Giovanni - AIRS Online Visualization and Analysis 



< ! ► <2* a | A ® http://gdatal.sci. gsfc.nasa.gov/daac-bin/G 3 /result 5 , cgi?w 5 id= 129113789115668 &app=l C Q7 Google 

Giovanni - AIRS Online Visualizat... 


moscow fires - Google Search 


AIRS Online Visualization and Analysis 

AIRS Global 1.0° x 1.0° Daily Level-3 Products 


Result #1 e * Result #2 0 * 


Results #3 O 


Remove All 


Visualization Results 


Download Data Product Lineage Acknowledgment Policy 


| Edit Piot Preferences [| Refine Constraints* | 
* Applies to the whole results set (all plots) 
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The Data Usage Cycle Refactored 


Select 



Visualize 




Acquire & 


Prepare 


OPeNDAP 
Subsetting 
Quality Filter 
Reformat 


Analyze 










Seamless Search and Analysis 


precipitation 



precipitation (mm hr-1) 
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Start: 2010-10-31 00:00:00 

End: 2010-10-31 23:59:59 
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Measurement: 


Soil Moisture (SMAP) 

Filter Quality?: 
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